\section{Evaluation}


\begin{tabular}{C{2.7in}C{2.3in}}

\begin{tabular}{|C{1cm}|C{0.9cm}|C{1.2cm}|C{0.9cm}|C{0.9cm}|}

\hline
{\small Feature} & \# & {\small Instruction} & {\small Control Flow} & {\small External} \\ \hline
{\small Idioms} & {\small 28963} & $\surd$ & & \\
{\small Graphlet} & {\small 16330} & $\surd $ & $\surd$  & \\
{\small Libcall} & {\small 11340} &  & & $\surd$ \\
{\small N-gram} & {\small 290217} &$\surd$  & &  \\
\hline
\end{tabular} 
&
\includegraphics[width=2.5in]{acc_features.pdf}\\

{\footnotesize Tab 1. Number of Each Feature} & 
{\footnotesize Fig 1. Acc-Features} \\
\end{tabular}

We extract our training data from git repository. Due to the time limit, we only apply our technique on the Dyninst [1] project. 

\subsection{Methodology}

In the first step, we use git blame command to get the authorship label for each line in each source file. 
There are several caveats here. One is that the committer may not the only author of the commit. A commit can be co-work of several people and committed by the committer. 
The other one is that the same author may have different account names doing commit. 
We have not handled the first issue. We handle the second issue manually by checking whether two accounts have similar names or asking the related committers.

In the next step, we compile the project with full debugging information and get a set of binary libraries. 
To transfer the author label we get in the first step to binary code level, we use line number interfaces in SymtabAPI [4] to see which source line the byte of code comes from. 
There are also several issues in this step. The first one is that a function may contain bytes from different author. 
We only keep the functions that an author exists who contributes no less than $95\%$ of the whole bytes. 
The second problem is that there are some bytes that are generated by the compiler or come from external library (like libc++). We exclude those bytes from our $95\%$ requirement. 
We discard those functions that do not meet our requirement.

In the third step, we extract features from the remaining functions coming from the second step and associate those features with the author label. 
Readers may notice that, in the remaining function, we do not discard the bytes generated by the compiler or bytes from external libraries. 
We expect that those bytes can still exhibit some properites of the author 
since the author's behavior can affect what external libraries to be included and what code to generate for the compiler.

\subsection{Result}

We extract 11782 functions from the Dyninst project. There are 44 authors that contribute at least one valid function. 
The data set is quite skewed. The first three authors ordered by the number of valid functions they contribute have 2583, 2187 and 1175. 
On the other side, there are 15 authors who have less than 10 functions in the data set. The features we extract summarize the characteristics of several code level, as shown in Fig. 1. We conduct 10 fold cross-validation for different feature combinations. The result is shown in Fig. 2.

%Fig 1 and Fig 2

As we can see in the Fig 2, we can achieve 62\% accuracy when using all feature types. It is surprising that using n-gram alone can give, 66\% accracy, even better results. We suspect this is caused by inproper parameter tuning.




